1. Field of the Invention
The present invention relates to a instruction recognition system having a gesture recognition function, and more particularly, to a system which performs user interaction.
2. Description of the Invention
A conventional practice has been to photograph a person through use of a camera, to detect the head of the person, and to perform predetermined processing on the basis of the recognition results.
For example, an interaction processing system and an interaction processing method, both being described in Japanese Patent Laid-Open No. Hei-9-218770, effects control of interaction by recognition of nodding action.
In a case where interaction is performed through mere recognition of nodding action, accurate detection of the nodding action is difficult. Since nodding action is performed sometimes without the person being aware that he is nodding, the system encounters difficulty in determining whether or not the nodding action or a like action truly reflects the user""s intention while the system awaits a user""s instruction. For this reason, the user""s instruction cannot be recognized correctly. Further, in a case where interaction is performed through mere nodding action, detailed interaction cannot be realized. Accordingly, interaction control that correctly reflects the user""s intention cannot be realized.
Accordingly, the object of the present invention is to provide a recognition system and an instruction recognition system, both of which can more correctly recognize the nodding action of a person and perform interactive control that closely reflects the user""s intention.
To this end, the present invention provides an instruction recognition system having a gesture recognition function for recognizing a user""s gesture, the system comprising:
a gesture recognition section for recognizing a user""s gesture;
a line-of-sight detection section for detecting the line of the user""s sight;
a display section for indicating image data; and
a controller which performs predetermined processing on the basis of recognition results realized by the gesture recognition section when the system awaits the user""s instruction while candidates for selection are indicated on the display section, provided that the line of sight detected by the line-of-sight detection section is determined to be oriented upward and that a predetermined gesture is recognized by the gesture recognition section.
While the system awaits a user""s instruction, if the line of sight detected by the line-of-sight detection section is determined to be oriented upward and the gesture recognition section recognizes predetermined gesture, predetermined processing on the basis of recognition results realized by the gesture recognition section is performed. The gesture is seen to correspond to the candidate for selection to which the line of sight is directed. Even when the system performs processing based on the gesture, processing correctly reflecting the user""s intention can be performed.
Preferably, the detected line of sight is determined to be oriented toward the candidate for selection on condition that the line of sight detected by the line-of-sight detection section corresponds to a gazing state where eyes remain stable for a predetermined period of time. In a case where the line of sight merely passes by, the system can eliminate the motion of the eyes irrelevant to the user""s intention, thus enabling execution of processing reflecting the user""s intention.
The present invention also provides an instruction recognition system having a gesture recognition function for recognizing the user""s gesture, the system comprising:
a nodding action recognition section which detects the user""s nodding action on the basis of a photographed image of the user""s head;
a line-of-sight detection section for detecting the line of the user""s sight;
a display section for indicating image data; and
a controller which determines selection of the candidate for selection, toward which the line of sight is directed, and performs predetermined processing when the nodding action recognition section detects nodding action and when the line of sight detected by the line-of-sight detection section is oriented toward any one of the candidates for selection during the nodding action, while the system awaits the user""s instruction and one or a plurality of candidates for selection are displayed on the display section.
While the system awaits a user""s instruction, the nodding action recognition section detects nodding action, and the line of sight detected by the line-of-sight detection section is determined to be oriented toward any one of candidates for selection during the nodding action. In a case where nodding action is detected and the line of sight is oriented toward any of the candidates for selection while the system awaits an instruction from the user, predetermined processing is effected, provided that the candidate toward which the line of sight is oriented is selected. As a result, processing is performed on the assumption that the nodding action is directed toward the candidate toward which the line of sight is directed, thus enabling execution of processing correctly reflecting the user""s intention.
Preferably, the gazing state corresponds to a state in which the line of sight is directed toward a predetermined candidate for selection from the beginning of the nodding action to the end thereof. As a result, the user""s intention of selection of the candidate to which the eyes of the user are directed can correctly be reflected on the processing.
Preferably, the nodding action recognition section recognizes nodding action when a certain portion of the user""s head moves vertically within a predetermined period of time and when the amount of vertical movement exceeds a predetermined distance. Consequently, the nodding action can be detected correctly.